Papers by Benjamin M. Mervak
CLEAR: A Clinically Grounded Tabular Framework for Radiology Report Evaluation (2025.findings-emnlp)
Copied to clipboard
Yuyang Jiang, Chacha Chen, Shengyuan Wang, Feng Li, Zecong Tang, Benjamin M. Mervak, Lydia Chelala, Christopher M Straus, Reve Chahine, Samuel G. Armato Iii, Chenhao Tan
| Challenge: | Existing metrics lack the granularity and interpretability to capture nuanced clinical differences between candidate and ground-truth radiology reports. |
| Approach: | They propose a tabular framework with E**xpert-curated labels and an attribute-level comparison for radiology report evaluation (**CLEAR) |
| Outcome: | The proposed framework can extract clinical attributes and provide automated metrics that are strongly aligned with clinical judgment. |
GPT-4V Cannot Generate Radiology Reports Yet (2025.findings-naacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are becoming multimodal, and GPT-4 models are supposed to possess advanced skills across a wide range of domains, including high-stakes scenarios such as medicine. |
| Approach: | They perform a systematic evaluation of GPT-4 in generating radiology reports across three chest X-ray report benchmarks: MIMIC-CXR, CheXpert Plus, and IU X ray. |
| Outcome: | The proposed model fails in lexical and clinical efficacy metrics . the distributions of model-predicted labels remain constant regardless of groundtruth conditions on the image, suggesting that the model is not interpreting chest X-rays meaningfully. |